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Abstract — In  this  paper,  we  analyze  the  information  theoretic 
aspects  of  the  phenomenon  of  stochastic  resonance  in  terms  of 
the  algebraic  properties  of  the  channel  matrix.  The  binary-input 
binary-output  channel  model  under  discussion  is  a  threshold 
system.  We  offer  an  algebraic  view  of  Kosko’s  Forbidden  Interval 
Theorem,  and  extend  his  result  in  certain  cases. 

Index  Terms — Stochastic  resonance,  information  theory,  for¬ 
bidden  interval  theorem. 

I.  Introduction 

In  this  paper,  we  analyze  the  information  theoretic  aspects 
of  the  phenomenon  of  stochastic  resonance  in  terms  of  the 
algebraic  properties  of  the  channel  matrix  for  a  simple,  but 
physically  meaningful,  threshold  based  system. 

The  communication  channels  under  study  in  this  paper 
are  binary-input  binary-output  discrete  memoryless  channels 
[11],  That  is,  the  channel  is  discrete,  memoryless,  and  has  a 
stationary  channel  matrix,  the  input  random  variable  X  has 
values  {t\,if\,  and  the  output  random  variable  Y  has  values 
{01,02}-  The  event  X  =  tk  is  the  event  that  the  symbol  tk  is 
input  to  the  channel,  and  the  event  Y  =  Ok  is  the  event  that 
the  channel  output  is  the  symbol  o/,;.  We  use  the  shorthand 
notation  of  a  (2,2)  channel,  respectively. 

We  use  the  (2, 2)  channel  to  study  the  phenomena  of 
stochastic  resonance  (SR)  that  arises  in  the  discrete  time 
threshold  neuron  model  as  first  described  by  [4],  A  nice 
introduction  to  this  is  given  in  [5,  Sec.  2],  However,  we  closely 
follow  the  mathematics  of  [2]  in  this  paper. 

Algebraic  information  theory  (AIT)  is  an  approach  to  Shan¬ 
non  information  theory  based  upon  the  algebraic  properties, 
such  as  the  determinant  and  eigenvalues,  of  the  channel  matrix 
(e.g.  [9],  [3] ).  This  idea  was  briefly  discussed  in  [13].  We  view 
Kosko’s  FIT  [5,  Thm.  1],  [6],  [7]  in  light  of  AIT. 


I-A.  Mutual  Information  and  Capacity  for  (2,2)  Channels 

The  channel  matrix  represents  the  conditional  probability 
relationships  between  the  input  and  output  symbols.  That  is, 
a  =  P(o1\t1),a=_P(o2\ti),b  =  P(oi|t2),  and  b  =  P(o2|t2), 
where,  in  general,  (•)  =  1  —  (•).  We  have  P(X  =  if)  =  x\  and 
P(X  =  if)  =  x2,P(Y  =  of)  =  yi  and  P(Y  =  of)  =  2/2- 
Since  ,:;:2  =  :c\  we  simplify  notation  further  and  set  a:  := 
Xi  and,  similarly,  y  :=  y  \ .  We  have  the  following  equivalent 
expressions: 

(2/1, 2/2)  =  (xi,x2)- ^  ,  (y,  y)  =  (x,  x) ■  (^b  ■  W 

and  the  channel  matrix  M,  where 

M,=  (“  f).  (2) 

We  see  that  the  (2,2)  channel  is  completely  described  by  the 
channel  matrix  M.  However,  since  M  is  a  stochastic  matrix, 
M  itself  is  completely  described  by  the  2-tuple  (a,  b).  Thus, 
a  (2,2)  channel  is  uniquely  identified  with  a  point  in  the  unit 
square  [0, 1]  x  [0, 1], 

Using  standard  Shannon  information  theory  [15]  we  have 
the  mutual  information 

I  =  H(Y)  -  H(Y\X)  (3) 

where  H{Y)  is  the  entropy1  of  the  random  variable  Y,  which 
for  the  (2,2)  channel  is  the  same  as  h(y)  and  H(Y\X)  is 
the  conditional  entropy.  Holding  (a,  b)  fixed,  we  have  that  the 

'Unless  noted  otherwise  all  logarithmic  bases  are  base  2,  where  log(a;) 
is  the  base  2  logarithm  of  x.  If  we  wish  to  use  the  natural  logarithm  we 
will  use  the  notation  In.  The  binary  entropy  function  is  denoted  as  h(x)  := 
—x  log(fr)  —  (1— x)  log(ic),  and  the  natural  binary  entropy  function  is  denoted 
as  he(x)  :=  —x  ln(ai)  —  (1  —  x)  ln(ai). 


mutual  information  given  by  Eq.  (3)  /  is  a  function  of  x, 
setting  f(x)  =  (a  —  b)x  +  b,  we  have  that 

I (x)  =  h  (/ (x))  —  xh(a)  —  xh(b) .  (4) 

The  capacity  C  of  the  (2,2)  channel  determined  by  the 
parameters  (a,  b)  is  the  well-defined  continuous  maximum  of 
the  mutual  information  ([16,  Eq.  5],  Ash  [1,  Eq.  3.3.5],  or  [8], 
[9]) 

CM)  =^MM»)+log  + 

/  ah(b)-bh(a )  bh(a)-ah(b )  \  (5) 

=  log  (2  +2  «-*  J 

where  C(a,  a)  :=  0.  Note  that  capacity  is  zero  if  and  only  if 
a  =  b  (concavity  arguments  show  that  for  all  x,  I{x)  =  0  iff 
a  =  b).  One  also  has  the  symmetries 


malize  those  results  into  the  notation  of  [7].  We  borrow  freely 
from  [13]  in  what  follows.  We  use  the  notation  (a,(3\2;0) 
to  specifically  denote  a  (2\2;0)  threshold  system  with  inputs 
a  or  /3  with  an  understood  additive  noise  distribution.  If  we 
wish  to  make  the  noise  distribution  explicit  we  will  write 

The  inputs  to  the  threshold  system  are  a  signal  X  taking 
on  the  values  a  or  /3.  The  (disturbance)  noise  is  added  to  the 
transmission  signal  resulting  in  the  received  signal  93: 

m  =  X  +  mv  .  (7) 


which  is  now  thresholded  resulting  in  Y 


Y  = 


if  <  6>; 
if  93  >  0. 


(8) 


C(a,  b)  =  C(b,  a)  =  <7(1  -  a,  1  -  b)  =  C{  1  -  b,  1  -  a)  .  (6) 

In  summary,  C(a,  b)  is  defined  and  continuous  for  all  (a,  b)  £ 
[0, 1]  x  [0, 1],  is  zero  iff  a  =  6,  and  is  1  if  and  only  if  (a,  b)  = 
(1,  0)  or  (0, 1),  with  the  symmetries  as  illustrated  in  Figure  1. 


Fig.  1.  Capacity  as  a  function  of  (a,  b). 


Fig.  2.  Capacity  level  sets 

The  level  sets  of  capacity  can  be  seen  in  both  Figures  1  &  2. 
To  solve  for  b  in  terms  of  a,  or  visa  versa,  for  a  given  level  set 
does  not  seem  to  have  a  closed  form  solution  due  to  the  high 
non-linearity  of  the  capacity  expression  involving  logarithms 
and  the  binary  entropy  function.  This  makes  working  with 
them  directly  rather  difficult,  hence  the  approximations  as 
developed  in  [11],  [12]. 

II.  The  (2\2\0)  Threshold  System 

The  (2\2;0)  threshold  system  is  a  binary-input  binary- 
output  discrete  time  threshold  system.  We  will  not  go  into 
detail  on  the  neuroscience  background  of  (discrete  time) 
threshold  systems.  We  point  the  interested  reader  to  [10].  We 
start,  though,  with  an  exposition  as  given  by  [2],  then  we  nor¬ 


II-A.  Shannon  Model  of  the  (01l\2;9;Af([i,<j2))  threshold 
system 

a2)  is  a  normal  distribution  with  mean  [i  and  variance 
(power)  <j2. 


M  = 


1  —  a 
1  -  b 


(9) 


The  inputs  to  the  channel  are  0  or  1,  and  the  outputs  are 
the  two  distinct  symbols  o\  and  02.  The  actual  output  values 
do  not  affect  the  information  theoretic  analysis  of  the  problem 
(the  input  values  do!). 

Following2  [2],  we  are  letting  (%>  be  J\f(0,  cr2^,  with 
probability  density  function  fa(t)  =  exp(^2-).  and 

cumulative  distribution  function  Fa{x)  =  fa{t)dt.  We 
denote  the  standard  normal  (  Af(0, 1)  )  cumulative  distribution 
function  as 

1  rx  _t2 

$(:e)  :=  . _  /  e~^~ dt  .  (10) 

V2ttJ~oo 

Change  of  variables  shows  that  Fa(x)  =  <&(x/cr).  We  take 
a  >  0,  and  look  at  the  limiting  situations  as  er  — ►  0+. 

The  input  random  variable  X  represents  the  physical  trans¬ 
mission  of  a  signal  X.  In  our  situation  X  takes  on  two  distinct 
values,  0  or  1.  The  disturbance  is  added  to  the  transmission 
signal  resulting  in  the  received  signal  93: 

93  =  X  +  Tis  .  (11) 


The  output  random  variable  is  determined  as: 


Y  = 


if  93  <  6»; 
if  93  >  0. 


(12) 


Therefore,  we  easily  have  [2]  that 
a  =  P{ 0  +  Af(0,  o2)  <9)  =  P(AT(0,  cr2)  <  0)  =  Fa{0),  and 
b  =  P(l+A/'(0,  a2)  <0)=  P(AT(0,  a2)  <  0-1)  =F<r(9- 1). 


2In  general,  the  noise  need  not  have  such  a  simple  form,  and  is  a  subject  of 
considerable  research  such  as  a  general  dynamical  model  in  terms  of  stochastic 
differential  equations  ([14]).  However,  this  paper  is  only  concerned  with  the 
zero  mean  normal  distribution  ( A/’(0,<72)  ). 


So,  the  channel  matrix  is  of  the  form3 


M  = 


a  1  —  a 
b  1-6 


Fa(0)  F„{0) 


MO-1)  Fa{0-1). 


*(£)  $(f) 


So  we  have  that  the  capacity,  in  closed  form,  is 

C{Fa{0),Fa{0-  1))  = 


(13) 


(14) 


lOg  2  F<r(e)-F„(fl-1) 

F0.(fl-l)-h(F0.(8))-F0.(8)-h(F0.(8-l))  \ 

+  2  F<r(8)-F„(fl-1  ) 

Unfortunately,  this  does  not  seem  to  be  a  very  tractable 
expression.  This  is  where  we  turn  to  algebraic  information 
theory.  Keep  in  mind  that 

C(F<r{0),Ftr(0-l))  =  c($(^)  ,$(^))  ■  (15) 

III.  Algebraic  Information  Theory 
First,  we  review  some  notation  and  results  from  [9].  We 
say  that  a  (2,2)  channel  is  nonnegative  if  a  >  b,  and  it  is 
positive  if  a  >  b.  The  negative  channels  are  those  with  a  < 
6,  and  the  zero  channels  are  those  with  a  =  b.  The  set  of 
zero  channels  is  identical  to  the  set  of  channels  with  zero 
capacity.  In  [9],  the  directed  complete  partially  ordered  set 
(dcpo)  of  compact  subintervals  of  the  unit  interval  with  the 
Scott  topology  I[0, 1]  is  studied.  The  nonnegative  (2,2)  channel 

(a,  6)  with  matrix  M  =  (^  is  uniquely  identified  (under 

a  monoid  isomorphism)  with  the  interval  [6,  a]  £  I[0, 1], 

Algebra  fact  1 

What  is  important  for  us  is  that  [9,  Thm.  4.9]  tells  us  for 
nonnegative  channels  that 

Theorem  3.1:  (AIT  ordering  theorem) 

a  <  a'  =>  C(a,b)  <  C(a',b),  and 


Fig.  3.  \a  —  b\  level  sets,  compare  to  level  sets  of  capacity  in  Fig.  2. 


It  is  worth  noting  that  the  level  sets  of  are  identical, 

up  to  the  level  set  ,  of  the  level  sets  of  |a  —  6|  (Fig.  3).  In 
essence  what  we  are  suggesting  is  a  linearization  of  channel 
capacity.  We  wish  to  use  this  approach  to  discuss  SR,  and 
compare  the  linear  approach  to  Kosko’s  FIT  [5,  Thm.  1],  [6], 
(7]. 


Also  keep  in  mind  that  for  nonnegative  channels 

(a  —  6)2 


2 ln(2) 


<  C (a,  b)  <  a  —  b  . 


(19) 


Therefore,  for  a  nonnegative  channel  analyzing  how 
det  M  =  a  —  b  changes,  gives  us  guidance  on  the  capac¬ 
ity  behavior.  Therefore,  we  can  approximate  the  non-linear 
behavior  of  capacity  by  the  linear  behavior  of  det  M,  in 
conjunction  with  the  AIT  ordering  theorem.  In  this  paper,  we 
will  concentrate  on  applying  the  AIT  ordering  theorem. 


IV.  Chapeau-Blondeau  Results  in  Light  of 
Algebraic  Information  Theory 


b’  <b  =>  C(a,b)  <C{a,tf)  .  (16) 

This  seemingly  obvious4  result  simply  follows  by  taking  the 
partial  derivatives  of  the  mutual  information. 

Algebra  fact  2 

Keep  in  mind  that  the  determinant  of  the  channel  matrix  is 
det  M  =  a  —  b  .  (17) 


The  following  fundamental  relation  reveals  much  about  capac¬ 
ity  in  terms  of  the  determinant.  It  is  shown  in  [11]  that 


(a-b)2 

2  ln(2) 


<  C(a,  b)  <  | a 


b\. 


(18) 


3Recall  x  =  1  —  x. 

4The  authors  of  [9]  were  surprised  that  this  result,  to  the  best  of  their 
knowledge,  did  not  exist  in  the  literature  prior  to  [9]. 
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Fig.  4.  Normal  distributions  /2  =  0,  with  cr  =  0.60  (blue),  a  =  1.0  (red). 


0=  0.60 

a  =A^q' 

/ 

2.5 

5 

-0 

.5 

0  0 

X 

5 

1 

5 

2 

Fig.  5.  Fa{6)  —  $>(6/cr)  with  a  —  0.60  (blue),  a  =  1.0  (red). 

It  is  interesting  that  there  is  ambiguity  surrounding  the 
formal  definition  of  SR  [10].  The  essence  of  SR  is  that 
increasing  noise  can  help  the  signal  to  noise  ratio  (SNR).  Since 
variance  is  the  power  of  a  normal  distribution  signal,  we  adopt 
the  following  definition  of  SR. 

For  a  (0, 1\2;  9;  J\f(0,  er2))  threshold  system,  we  denote  the 
Shannon  channel  capacity  with  9  fixed  and  variable  er  as 

Ce(a). 

Definition  4.1:  We  say  that  a  (0, 1| 2;  9;  Af(0,  er2))  threshold 
system  exhibits  SR  (as  a  function  of  a)  iff  there  exists  0  <  <7l 
and  er2  such  that  o\  <  cr2  and  C'e(o'i)  <  Cg(cr2). 

This  goes  along  with  the  concept  of  adding  noise  (a2  is 
the  power,  and  er  =  0  corresponds  to  no  noise)  to  amplify  the 
intended  signal. 

In  [2],  and  in  more  detail  in  [13],  it  is  graphically  demon¬ 
strated  and  discussed  that  for  0  <  9  <  1  there  is  no  SR,  but 
for  8  <  0  or  8  >  1  there  is  SR  as  er  goes  from  zero  to  infinity. 
Kosko’s  FIT  [5],  [6]  can  be  used  to  prove  this,  but  let  us  take 
an  algebraic  approach  which  we  will  later  use  for  an  algebraic 
proof  of  the  FIT  in  general. 

In  Figure  6  (see  [13,  Fig.  3]  we  see  that  there  is  no  SR  for 
9  £  {.8,  .9,  .95,  .99, 1},  and  there  is  SR  with  a  single  hump 
for  6  £  {1.01,1.05,1.1,1.2}. 


Fig.  6.  Capacity,  for  6  £  {.8,  .9,  .95,  .99,  1, 1.01,  1.05, 1.1,  1.2},  as  a  function  of  a. 


Theorem  4.2:  A  (0, 1|^;  0;  A/”(0,  cr2))  threshold  system  ex¬ 
hibits  SR  if  and  only  if  9  [0, 1]. 

Proof:  It  is  shown  in  [13,  Thml.l]  that 

C.5 +d(&)  =  C.5-(z(ct).  Therefore  it  suffices  to  show  the 
threshold  system  does  not  exhibit  SR  for  9  £  [.5, 1],  and  that 
it  does  exhibit  SR  for  9  >  1.  (See  Figs.  4  &  5). 

ONLY  IF  part  4= 

—  For  9  £  [.5, 1) 

Since  a  =  &(9/cr)  we  see  that  since  9  >  0,  a  decreases  as  er 
increases.  On  the  other  hand,  b  =  $(0  —  1)  increases,  since 
9—1  <0.  Since  a  decreases  and  b  increases,  the  AIT  theorem 
tells  us  capacity  decreases  as  o  increases. 

—  For  9=1 

As  above,  a  decreases  as  er  increases.  However,  b  =  F’o-(O)  = 
1/2  for  all  er.  Since  a  decreases  and  b  is  constant  the  AIT 
theorem  still  tells  us  capacity  decreases  as  er  increases. 

Thus,  there  is  no  SR  for  9  £  [.5  :  1], 

IF  part  => 

—  For  9  >  1 

In  this  situation  both  a  and  b  decrease  as  er  increases,  so  we 
cannot  apply  the  AIT  theorem.  However,  since  $(00)  =  1,  we 
see  that  as  er  — y  0+  that  both  a  — >  1  and  b  — >  1,  so  capacity 
approaches  zero.  Since  there  exist  er  >  0  such  a  =  Fa(9)  f 
Fa(9  —  1)  =  b,  we  that  that  capacity  (which  is  continuous)  is 
increasing  for  some  interval  of  er. 

Thus,  there  is  SR  for  9  >  1  ■ 

Corollary  4.1:  For  9  £  [.5,1),  capacity  decreases  from  a 
limiting  value  of  1  down  to  a  limiting  value  of  0,  and  for 
9  =  1,  capacity  decreases  from  the  limiting  Z-channel  capacity 
of  «  .3219  down  to  a  limiting  value  of  0.  In  addition,  for  9  >  1 
the  plot  of  capacity  will  exhibit  at  least  one  hump  (see  Figure 
6). 

Proof: 

—  For  9  £  [.5,1) 

As  er  — >  0+,  (a,  b)  — >  (1, 0),  so  C@(0)  —y  1. 

As  er  — >■  00,  (a,  b)  —y  (1/2, 1/2),  so  Cg(oo)  — >  0. 

—  For  0  =  1 

As  er  0+,  (a,  b)  =  (1, 1/2),  so  C'e(O)  ->•«  .3219. 

As  er  — >  00,  (a,  b)  — >  (1/2, 1/2),  so  Cg(oc>)  — >  0. 

—  For  9  >  1 

As  above,  as  er  — >  0+,  both  a  — >  1  and  b  — >  1,  so  capacity 
approaches  zero.  As  er  — >  00  both  a  — >  1/2  and  b  — >  1/2, 
so  capacity  approaches  zero.  Therefore,  there  is  at  least  one 
“hump”  in  the  plot.  ■ 

V.  Kosko  Bipolar  Threshold  System 

Say  we  have  a  (0, 1\2;  91;  Af(0,  er'2))  threshold  system. 
Consider  a  normalization  such  that  9  =  29'  —  1  and  er  =  2 er', 
and  let  the  channel  inputs  {-1,1}  replace  {0,1}  respectively. 

The  behavior  of  (0, 1|^;  9' \  A/”(0,  er'2))  threshold  system  is 
the  same  as  a  (—1, 1\2;  8;  Af(0,  er2))  threshold  system,  where 
9  =  29'  —  1  and  a  =  2 a' .  This  means  we  identify  the  closed 
interval  [0, 1]  with  [—1, 1]  under  the  mapping  x  — >  2x—  1  and 


we  analyze  (—1, 1| 2;  20'  —  1;  A/"(0,  2a'2))  and  the  SR  regions 
are  the  same. 

So  for  a  (—1, 1| 2\  20'  —  1 ;  A/”(0,  2a'2))  threshold  system  we 
have 

a  =  P(-l+J\f(0,a2)  <  6)  =  P(J\f(0,  a2)  <6  +  1)  = 
Fa(0  +  1)  =  F2a'( 26')  =  $(0'/a'),  and 
b  =  P(  1+  AA( 0,  a2)  <0)  =  P(U(0,  a2)  <6-  1)  = 

Fa(6  -  1)  =  F2a,(20’  -  2)  =  $  {(O’  -  l)/a'). 

So,  the  channel  matrix  is  of  the  form 


a  1  —  _  I F„(0  +  1)  Fcr{6  +  1)^ 

b  l~b)  \Fa{6- 1)  Fa(6-l)J 


Therefore,  we  see  that  the  channel  matrix  for  the 
(— 1, 1\2;  6;  A/”(0,  cr2))  threshold  system  is  the  same  as  the 
channel  matrix  for  the  (0, 1| 2;  0'\ A/”(0,  cr'2)),  up  to  a  normal¬ 
ization.  In  fact,  there  is  nothing  special  about  ±1,  so  let  us 
use  ± /l .  ,4  >  0  for  the  X  inputs  to  the  system  (and  channel). 

For  a  (— A,  A\2;  0;A/”(O,  a2))  threshold  system,  we  denote 
the  Shannon  channel  capacity  with  6  fixed  and  variable  a  as 
Cg(a). 

Definition  5.1:  We  say  that  a  (— A,  A\2;  6;  Af(0,  a2)) 
threshold  system  exhibits  SR  (as  a  function  of  a)  iff  there 
exists  0  <  (7i  and  a2  such  that  cri  <  a2  and  Cg(ai)  <  Cg(a2). 

Theorem  5.2:  If  we  map  the  interval  [0, 1]  —>  [— A ,  A] 
under  the  affine  map  x  —>  A( 2x  —  1)  and  set  6  := 

A(26'  —  1),  and  a  :=  2Aa' ,  then  the  channel  matrices,  as 
functions  of  a'  and  O',  for  the  (0, 1\2;  O';  A/"(0,  a'2))  and 
(— A,  A\2;  0;  Af(0,  a2))  threshold  systems  are  the  same.  Fur¬ 
thermore,  the  (— A,  A\2;  0;  A/”(0,  cr2))  threshold  system  ex¬ 
hibits  SR  if  and  only  if  0  {—A,  A],  and  its  capacity,  as 

a  function  of  a,  exhibits  a  symmetry  about  0. 

Proof:  For  the  (— A,  A\2;  6;  A/"(0,  a2))  threshold  system 
a  =  p(-A+Af{0,a2)  <  0)  =  P(Af(0,a2)  <  0  +  A)  = 
Fa(0  +  A)  =  F2Aa,(2A0’)  =  §(0'/a'),  and 
b  =  P(A  +  AA(0,  a2)  <  0)  =  P(Af(0,  a2)  <0-  A) 

=  Fa(0  -A)  =  F2A„'(2A0'  -  2A)  =  $  ((0'  -  1  )/a'). 

So,  by  Thm  4.2,  the  (— A,  A\2;  6;  A/”(0,  a2))  threshold  system 
exhibits  SR  if  and  only  if  6'  [0, 1],  which,  since  O'  = 

is  equivalent  to  0  (j  [—A,  A]: 5  ■ 

V-A.  Non-zero  Mean 

Up  to  now  we  have  been  looking  at  how  6  relates  to  the 
interval  [—A,  A]  with  additive  disturbance  A/"(0,  cr2)).  Now  let 
us  consider  additive  disturbance  with  non-zero  mean,  that  is 
A f([i,  a2)).  For  a  (— A ,  A\2;  0; Nf(n,  a2))  threshold  system  we 
denote  the  Shannon  channel  capacity  with  0  fixed  and  variable 
a  as  C0 (a). 

5At  this  point  we  are  not  concerned  with  which  standard  deviation  value 
maximizes  capacity,  so  we  do  not  concern  ourselves  with  the  distinction 
between  cr  and  a'.  We  include  this  footnote  to  assure  the  reader  that  we 
have  not  forgotten  about  the  normalization. 


Definition  5.3:  We  say  that  a  (— A,  A\2;  6;  Af(n,  a2)) 
threshold  system  exhibits  SR  (as  a  function  of  a)  iff  there 
exists  0  <  a i  and  a2  such  that  cri  <  a2  and  Cg(o\)  <  Cg(a2). 


Consider  the  map  x  A(2x  —  1)  +  /r  and  set 
6  :=  A( 20'  —  1)  +  fi,  and  cr  :=  2 Aa'. 

So,  for  a  (— A ,  A\2;  0;Af(/j,,  a2))  threshold  system,  we  have 
that  a  =  P(— A  +  A f(n,  a2)  <  0)  =  P{N(n,  a2)  <  0  +  A)  = 
$  =  $  (O' /&')■>  and  similarly  b  =  $  = 

$  (( 0 '  —  1  )/a').  So  there  will  be  SR  if  and  only  if  O'  ^  [0, 1], 
However,  since  O'  =  6+2(f'i,  this  is  equivalent  to 

Theorem  5.4:  The  (— A,  A\2;  0;  Af(n,  a2))  threshold  sys¬ 
tem  exhibits  SR  if  and  only  if  0  ^  [fi  —  A, /j  +  A],  or 
equivalently,  /r  ^  [0  —  A,  0  +  A). 

We  consider  Kosko’s  FIT  [5,  Thm.  1],  [6],  [7]  in  light  of 
AIT.  Note,  his  region  of  interest  is  fi  [0  —  A,  0  +  A]  . 


V-B.  Kosko  Formulation 

We  use  [6,  Thm.  1.1,  Thm  1.2]  for  Kosko’s  FIT. 

1)  Instead  of  our  Eq.  (8),  Kosko  has  the  slightly  different 


Y  = 


if  SK  <  0; 
if  m  >  0. 


(21) 


This  has  the  effect  of  changing  the  closed  interval 
[0  —  A,  0  +  A]  to  the  open  interval  (0  —  A,  0  +  A). 


2)  Kosko  requires  the  input  signal  to  be  subthreshold;  that 
is  A  <  0.  We  do  not. 


3)  We  restricted  the  disturbance  noise  to  be  Gaussian, 
Kosko  allows  much  more  freedom,  and  in  fact  studies 
non-finite  variances.  We  will  address  this  in  future 
work.  Please  note  that  the  “If”  part  of  our  proof  is 
essentially  the  same  as  Kosko’s  so  there  should  be  no 
problem  with  non-Gaussian  distributions  (at  least  those 
with  finite  variance). 


4)  The  “Only  If”  part  our  our  proof  relies  on  AIT.  We 
show  that  the  capacity  is  monotone  decreasing.  Kosko 
shows  that  capacity  decreases  from  the  maximal  value 
(obtained  from  the  limit  when  a  — >  0+),  but  he 
does  not  show  that  there  are  no  local  maxima.  This  is 
an  important  distinction.  This  is  why  our  generalized 
definition  of  SR  is  as  given  in  Def.  5.3. 

VI.  Capacity  Approximations 


In  [11],  [12],  [13]  we  showed  how  capacity  of  the  channels 
under  study  in  this  paper  can  be  simply  approximated  and 
bounded  by  simple  functions  of  the  determinant  of  the  channel 
matrix.  For  the  (— A,  A\2;  0; Nf(p,  a2))  threshold  system  we 
have; 


2  ln(2) 


$ 


0  +  A  —  /r 


-  $ 


0  —  A  —  n 


< 


Ca(0)  <  $ 


1  +  A  -  fj, 

a 


-  $ 


'  —  A  —  n 

a 


(22) 


We  want  to  see  when  these  lower  and  upper  approximations 
exhibit  SR.  That  is  simply  done  by  taking  the  first  derivative 
with  respect  to  a  and  setting  it  to  zero.  Since  fk(a  —  b)2  = 
2 k(a  —  b)  ■  j-{a  —  b),  and  a  >  b.  we  see  that  it  suffices  to 
consider  £  (<l>  -  $  (*=£=*))  =  0. 


6-\-A—p 


)-*( 


e~A~^  ) )  =  o. 


—  ($(  6  +  A  ~  M 
da  V  V  a 


9  —  A  —  /j, 


determinant  bounds  can  easily  give  us  the  regions  where  SR 
occur. 

In  future  work,  we  will  analyze  non-Gaussian  distributions, 
and  see  if  our  algebraic  approach  can  be  applied  to  more 
complex  systems.  In  particular,  we  want  to  see  if  our  algebraic 
results  for  (2,3)  channels  [12]  can  be  applied  to  similar 
threshold  systems. 

Acknowledgments 


e  2  dt  = 


da  yy/2TT  J »-A-„  J 

1  \-(9  +  A-n)  ~(g±4^)2  -(0-A-n) 


Setting  the  above  to  zero  we  arrive  at: 

9  T  A  fi  (o+A—^i 

9  —  A  —  n 

9  —  (/Li  —  A)  4A(8-n) 

=  e  2»2 


(8  +  A-m)2-(8-A-m)2 


9  —  (/i  + A) 


Provided  that  the  LHS  of  Eq.  (24)  is  positive,  which  is  true 
iff  6  \n  —  A,  /i  +  A],  we  arrive  at 


9  +  A  —  fi 
9  —  A  —  /x 


4  A{9-n) 
2^2 

2  A{9-n) 

\n(e-^-A) 

m{e-^+A) 


Provided  that  the  RHS  if  the  above  is  positive,  we  have 


If  9  >  n  +  A)  then  >  1,  if  9  <  fi  —  A)  then 

0  <  gZ(^+^4)  <  1-  Therefore  Eq.  (27)  has  a  solution  iff 
9  [/i  A,  fi  T  A\. 

Definition  6.1:  We  call  the  cr  derived  from  Eq.  (27)  the 
critical  sigma  approximation,  and  denote  it  as  ag-r. 

We  note  that  in  [13]  we  have  given  graphical  evidence 
for  how  well  the  algebraic  determinant  bounds  approximate 
capacity,  and  also  how  well  as~r  corresponds  with  the  actual 
sigma  that  locally  maximizes  capacity. 

What  is  most  interesting  is  that  one  can  obtain  ag-r  iff  the 
constraints  of  the  FIT  hold,  that  is  9  ^  [/i  —  A,  fi  +  A],  or 
equivalently  fi  ^  \0  —  AJ)  •  ,4].  Therefore,  our  approximating 
capacity  by  the  determinant  bounds  gives  us  another  form  of 
the  FIT,  which  is  derived  solely  from  the  algebraic  properties 
of  the  channel  matrix.  This  holds  promise  for  approximating 
when  more  complex  threshold  systems  exhibit  SR. 


VII.  Conclusion  and  Future  Work 

We  have  amplified  the  definition  of  stochastic  resonance, 
and  shown  how  an  algebraic  approach  to  capacity  and  the 
associated  issues  of  SR  is  a  valid  approach.  We  have  given 
an  alebraic  proof  for  part  of  the  FIT.  We  have  shown  how 
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